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1 Executive Summary 


Computer generated information displays provide a promising technology for offsetting the 
increasing complexity of the National Airspace System. To realize this promise, however, 
we must extend and adapt the domain-dependent knowledge that informally guides the de- 
sign of traditional dedicated displays. In our view, the successful exploitation of computer 
generated displays revolves around the idea of information management, that is, the iden- 
tification, organization and presentation of relevant and timely information in a complex 
task environment. 

The program of research described below leads to methods and principles for information 
management in the domain of commercial aviation. The multi-year objective of the proposed 
program of research is to develop methods and principles for determining task dependent 
interface content. These general methods and principles will. 


• Contain a clear partition between the contributions of the aviation task, 
specific aircraft equipment, human perception and cognition, and informa- 
tion management software to the design of an information display format. 
Because of this partition, the methods lend themselves to straightforward modifica- 
tion in response to changes in aircraft equipment, making it possible to generate good 
displays without depending upon direct operational experience. 


• Be represented as a computational system. A computational system ensures 
that the methods are executable, minimizes the role of unspecifed intuition, and en- 
sures the completeness of the methods for the cases covered. Questions arising in the 
early implementations of the system help to focus experimental questions and aid in 
the intepretation of experimental results. When validated, the computational system 
can serve as an automated aid for interface design. 


A prominent feature of the proposed methods and principles for determining interface con- 
tent is the role of human intention in identifying relevant aspects of a task environment. 
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Geddes (1989) equates human intention with symbolic plans that guide both behavior and 
information search. Different plains correspond to alternative methods and tools for address- 
ing what is apparently the same situation, and hence potentiadly identify different pertinent 
information. The use of plans to guide the identification of information requirements is 
referred to here as the plan-based approach. 

The hypothesized role of plans as the basis of a method for identifying information require- 
ments suggests a multi-year research program with the following objectives: 


1. Refine the plan decomposition and parameter determination procedure, and establish 
methods for achieving useful decompositions and parameter values. 

2. Empirically evaluate the feasibility of a plan based approach for the presentation of 
information, with respect to cognitive and engineering criteria. 

3. Define and refine the grain size of plan-based information, and parameters for display, 
with respect to human interpretive processes. 

4. Maintain and modify a computational system that translates plans into their infor- 
mation requirements. 

1.1 Summary of Accomplishments at Six Months 

The objective of the first year was to empirically evaluate the feasibility of a plan- based 
approach to the presentation of information. Our progress during the first six months with 
is briefly stated below, and discussed in more detail in section 4.0. 


• Refine the plan decomposition and parameter determination procedure, and establish 
methods for achieving useful decompositions and parameter values. 

- A plan and goal graph for a part of the domain of commercial aviation has been 
developed. The current heuristic for decomposing plans, based on the differing 
requirements for information, was applied and continues to be appropriate. 

• Empirically evaluate the feasibility of a plan based approach for the presentation of 
information, with respect to cognitive and engineering criteria. 

- The newly developed plan and goal graph demonstrates the relevance of the plan 
and goal graph concept for commercial aviation. 

- Relevant literature from Human Factors, Psychology, Computer Science, Engi- 
neering, Decision Sciences has been assembled and distilled into rules linking task 
properties and display requirements. 
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- A prototype set of presentation elements that provide for manipulation by an 
information manager has been constructed. 


In the following section we identify critical issues regarding the management of information 
that must be resolved in order to develop the desired methods. This introductory informa- 
tion is followed by a detailed report on the progress achieved during the first six months of 
the program. 
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2 Statement of the Problem 


The development of methods and principles for determining task dependent interface con- 
tent for aviation operations is not a novel objective within the human factors community. 
The availability of computer generated displays and the increasing complexity of the Na- 
tional Airspace System have made this an important objective for quite some time. 

Previous work conducted by Shalin and Geddes in this general area is discussed in detail in 
Appendix A. The following statement of the problem is developed in terms of the essential 
features of this work, revolving around the engineering goal of information management. 
the identification, organization and presentation of relevant and timely information in a 
complex task environment. Key issues include: The chosen unit of task analysis, cognitive 
theory regarding the selection and organization of information in real-world tasks, the ap- 
propriate grain size on both tasks and information, information management policies, and 
the substitution of domain-dependent common sense for insufficiently articulated display 
design methods and principles. 


2.1 Plans as the Unit of Task Analysis 

Webb, Geddes & Neste (1989) pointed out the problem of the chosen unit of task analysis. 
Webb et al. noted that the traditional unit of analysis is typically the scenario or situation. 
Since scenarios are too numerous to cover exhaustively, they are sampled for the purpose of 
display design, with the hope that displays designed with respect to the sampled scenario 
will transfer properly to unsampled scenarios in the operational environment. However, 
a given scenario may actually consist of a number of conceptually independent activities 
(e.g., handling an abnormal condition, contacting the company and flying the plane). A 
design procedure based on sampling a particular combination of activities risks permanent 
over-specialization of the displays to a relatively unique situation. Rather than risk this 
permanent over-specialization, Webb et al. recommend associating information with the 
individual primitive activities, and letting an intelligent information manager combine on- 
line the information to support the actitivities for a particular situation. 

Geddes suggests that these activities are ideally regarded as plans addressing concurrent 
intentions (Geddes, 1989; Howard, Hammer & Geddes, 1988; Rouse, Geddes & Curry, 1987; 
Shalin, Geddes, Miller, Hoshstrassser, Levi & Perschbacher, 1990; Shalin, Miller, Geddes, 
Hoshstrasser, Levi & Perschbacher, 1990; Webb, Geddes & Neste, 1989). Plans consist of 
different operational methods, with different tools to achieve the same goal (Miller, Galanter 
& Pribram, 1960; Sacerdoti, 1977). The distinction between plans and goals as elements 
that comprise intentions allows for different operational methods — with different tools, and 
hence different important associated information— to achieve the same intent. Accordingly, 
the large set of information in the situation can potentially be trimmed down by knowing 
which of the alternative plans are in use. 
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Domain plans and goals at various levels of abstraction axe organized by the plan and 
goal graph (Rouse, Geddes & Hammer, 1990; Sewell & Geddes, 1990). The plan and 
goal graph (PGG) represents a decomposition of the most abstract purposes of a system 
into increasingly resolved descriptions, until the descriptions are completely composed of 
primitive (i.e., directly executable) actions. Multiple uses (or parents) of plans and goals 
results in a directed acyclic graph. 

A portion of the plan and goal graph in the domain of commercial aviation, developed dur- 
ing our the first six months of research, is illustrated in Figure 1. The graph represents a 
decomposition of the most abstract purposes of a system into increasingly resolved descrip- 
tions, until the descriptions are completely composed of primitive (i.e., directly executable) 
actions. Plans are represented by rectangles and goals are represented by ovals. 



Figure 1. A portion of the plan and goal graph analysis for commercial aviation. 


2.2 The Selection and Organization of Compatible Information 

Structuring the task environment according to the activity one intends to perform is the 
essence of ecological approaches to perception (Gibson, 1979; Vicente, 1990, Vicente & 
Rasmussen, 1990.) The real-world cognition involved in flying an airplane depends essen- 
tially upon sampling and organizing information from an indefinitely large context (Geddes, 
1989). The available information can be selected for relevance to the current plans, thereby 
inverting the typical claim from within the Cognitive Science community about the rela- 
tionship between problem information and the knowledge structures in human memory. 
Typically an analysis of the information in the problem statement in limited laboratory 
or schoolbook tasks guides the definition of cognitive structures (Chi, Feltovich & Glaser, 
1981; Larkin, 1982; Greeno, 1983; McDermott & Larkin, 1978; Paige & Simon, 1966, ). 
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However, we ought to be able to examine cognitive structures defined, on some other basis, 
to guide the identification of related features of the external problem situation (Chase & 
Simon, 1973). 


Having hypothesized plans as the appropriate unit of analysis, information for display there- 
fore corresponds to tests of features of the environment that aid in the selection, execution 
and monitoring of plans to alter an undesirable state, or cause a desirable state to persist. 
Hypothesized information requirements for the Autopilot Speed Capture plan are shown in 
Figure 2. 1 


2.3 Plan Grain Size 

Adopting plans and their arguments as a framework for determining plan information needs 
does not define a complete approach to information management. Methods and principles 
for determining task dependent interface content require a proper grain size on plan descrip- 
tions so that they capture differences in the information associated with different plans, as 
well as the cognitive phenomena associated with the interpretation of information. More- 
over, once a grain size is determined, cognitive compatibility issues must now be considered 
for multiple concurrent or sequential plans, alternative plans, and the computational de- 
mands on the information management process. 


2.4 Information Management Policy 

The introduction of information management software makes explicit the potentially tacit 
notion of an information management policy governing display configuration. Information 
management policies should be formulated in response to the particular information prob- 
lems that are expected in the application domain. The kinds of problems associated with 
information have been classified by LaRC as the following: 


• information access 

• information perception 

• information interpretation 

• information deprivation 

• information application to task 

• information prediction 


J The intended connotation of requirement is closer to LaRC’s concept of desirement than the word implies. 
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The information management policy under investigation by this project addresses prob- 
lems of access, perception and interpretation. It can be described as a “greedy” approach 
that seeks to provide as much important information across all concurrent plans as can 
be presented, while tailoring the presentation form and location to facilitate perception 
and interpretation. This policy is particularly well suited for task domains in which the 
information user is generally well-supported with available information and well-trained in 
its use, but is under time constraints to access, perceive and interpret the information. 
This circumstance is common in aviation, but the policy may not deal with all information 
problems in the domain. 


Goal: Observe Speed Restriction 
Plan: Autopilot Speed Capture 
Set the autopilot mode 
Arm the speed capture 
Set the desired speed 
Monitor engine parameters 
Monitor speed change 


Information Requirements: 

Autopilot mode 

Autopilot status 

Current speed 

Speed to be captured 

Current altitude 

Altitude at capture 

Altitude remaining to capture 

Vertical speed 

Current DME 

DME at capture 

Distance remaining to capture 

Navigation radial at capture 

Navigation facility defining speed restriction 

Engine thrust setting 

Spoiler extension state 

Pitch attitude 


Figure 2. A plan and its information requirements. 


The information management policy incorporated into an information management system 
impacts the methods for identifying task information requirements because the dimensions 
manipulated by the information management system must be supported by an appropriate 
set of attributes associated with the information for display. The following parameter values 
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are manipulated under the current information management policy, to constrain the display 
of the information to be compatible with pilot reasoning. 


• The importance parameter — indicates the relative importance of the information, 
given limited display real estate. 

• The bandwidth parameter — indicates how frequently the value of an information ele- 
ment changes by an operationally meaningful amount, and hence, how frequently its 
displayed value will require updating. 

• The resolution parameter — indicates the requirements for fine distinctions between 
states. 

• The scope parameter — indicates the range of possible states that must be in view at 
any one time. 

• The control parameter — indicates the pilots need to control the value of an information 
element, constraining the display of the information to be coincident with an input 
device. 


Some of these parameters have already been manipulated in the design of dedicated displays. 
For example, the resolution on the dial for Indicated Airspeed is unevenly represented on 
the MD80, with the distance and number of tick marks between 250 and 300 equal to 
the distance and number of tick marks between 240 and 250. Abbott (1989) describes a 
display element developed by Boeing for 757/767 engine parameters that combines a dial 
without tick marks and a digital display, to complement the tradeoffs in scope, resolution 
and bandwidth that each component embodies. In both cases of display element innovation, 
we presume the designer was attempting to accomodate very different activities with one 
device. With computer generated display elements, we now have the option of adjusting 
the display element in question for the task at hand. 

Figure 3 illustrates the some of the parameter values for the Autopilot Speed Capture 
Plan. In contrast, if we were displaying information for a different plan to achieve the 
same goal, these parameters might change. For example, the pilot might Observe the Speed 
Restriction by effecting a Manual Power Change. The essence of this new plan is controlling, 
rather than monitoring, which is known to place different demands on pilot information 
processing (Harris k Spady, 1985; Abbott, 1989). We would make several adjustments in 
the parameters controlling the presentation of current speed information, increasing both its 
resolution and bandwidth, while reducing the parameters describing Autopilot information. 
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Goal: Observe Speed Restriction 
Plan: Autopilot Speed Capture 


Information Requirements: 

Autopilot mode 

importance 10 

scope 3 

resolution 3 

bandwidth 2 

control 10 

Autopilot status 

importance 10 

scope 2 

resolution 1 

bandwidth 8 

control 0 

Current speed 

importance 10 

scope 4 

resolution 7 

bandwidth 4 

control 2 

Speed to be captured 

importance 10 

scope 2 

resolution 7 

bandwidth 2 

control 10 

Current altitude 

importance 5 

scope 5 

resolution 2 

bandwidth 3 

control 1 


Figure 3. A Selection of parameterized information requirements. 

We can imagine the need to add to the existing dimensions of the information management 
problem. For example, information management based on information timeliness or age 
is not a part of the current information management policy. Of course, adding such a 
dimension to the information management system necessitates changes to the methods of 
analyzing plans for their information requirements. 
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2.5 Information and Presentation Element Granularity 

The information elements in Figure 3 indicate the contents and display parameters for 
what is to be displayed. They do not, by themselves, define a specific format. Formatting 
is accomplished by matching capabilities of specific presentation elements with the contents 
and parameter values of information elements. Information management policy is neces- 
sarily reflected in the granularity of presentation elements; The granularity of presentation 
elements sets an upper limit on the ability of the information manager to adapt the presen- 
tations to meet the information needs of specific tasks. In addition, presentation elements 
are characterized by a set of attributes that represent the available manipulations of the 
information manager. The granularity of these attributes limits the extent of information 
management. Coordination of the granularity of the presentation elements and their at- 
tributes with the information requirements and attributes is essential to realizing a visible 
effect of information management in the final display; A mismatch in granularity results in 
needless computation. 


2.6 Towards a Theory of Task-sensitive Display Design 

The persisting role for the operationally knowledgeable display designer (also noted by Ab- 
bott, 1989) may be regarded as a clue that domain-dependent common sense is informally, 
and indeed often successfully, substituting for insufficiently articulated display design meth- 
ods and principles. It is our hypothesis that the ability of the operationally knowledgeable 
designer to successfully develop display designs is due at least in part to a well developed, 
internalized set of operational principles or domain theory. One part of this domain the- 
ory links the task, equipment and human cognition to abstract, parameterized information 
requirements. This part of the domain theory is depicted in Figure 4. A second part of 
this domain theory exercises implicit knowledge about the perception and interpretation 
of display media features to link parameterized information requirements to presentation 
elements. Analysis of the underlying phenomena of perception and cognition of display 
features is expected to depend on a similar domain theory for presentation elements that 
predicts the interaction of an observer with a set of display features. 

In both cases, the domain theory provides an explanation of existing information and pre- 
sentation elements in terms that the information manager can use. While the domain 
theory does not serve as a generative mechanism for novel information or presentation ele- 
ments, it is able to generate explanations for unanticipated elements. Although respecting a 
similar — but tacit — domain theory, the operationally knowledgeable designer may be unable 
to predict the implications of new systems, and in any case, is subject to the ordinary human 
limi tations in such complex reasoning. Moreover, continued reliance on a knowledgeable 
designer allows this underlying theory to remain tacit, and impossible to test scientifically. 
The risk of an untested, unarticulated theory is the development of inconsistently successful 
methods, with poor transfer to new situations. 
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Bandwidth Scop« Resolution 



Figure 4. Attributes of information that determine parameter values for information 

management. 


2.7 Implications of Previous Work for the Development of Context Sen- 
sitive Displays 


Existing methods developed by Shalin & Geddes are plan-based, account for some of the 
cognitive processes of the pilot, as well as the dimensions of information management used 
by the system, and are formalized in a computational system that explicitly represents 
the domain theory. As illustrated in Figure 4, these methods make use of low-level task- 
dependent and task-independent pararmeters that describe the behavior of a particular 
piece of information in the context of a task. Thus the determination of task sensitive high- 
level parameter values for information (i.e., Scope, Resolution, Bandwidth and Control) has 
a sound theoretical foundation captured by the figure, but requires validation and testing 
with commercial pilots in an operational context. 
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The link between presentation elements and their parameter values has been the subject 
of considerable theoretical and empirical study (Spoehr & Lehmkuhle, 1982). The primary 
deficit of this work for the present purposes is that it has not been integrated into a set 
of coherent, rule-based claims about pilot performance in an operational context. Work on 
this task was initiated in Year 1 of this project. 

Note that we have not identified the correctness of the domain theory as a critical assump- 
tion in this project. Rather, the critical feature is to have any theory that has empirical 
entailments (Carroll, 1991). The theory can always be modified in fight of new findings, 
and the relevance of new findings is clarified by the context of the existing theory. 
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3 Approach 


First, we must operationalize our existing theoretical analysis of the relationship between 
displays and plans, by developing an appropriate empirical methodology. The existing 
methods are now being validated in Year 1, by using them to define an interface and produce 
a knowledge base for an information management system, and examining part-task pilot 
performance. Second, we must extend the empirical methodology to new part-task studies, 
to validate its adequacy. This constitutes the basis of the Year 2 work, using the method- 
generated display, and variations in information management policy. Finally, we must 
apply the methodology to demonstrate the performance impact of plan-based information 
management. This evaluation will be planned during Year 2 and conducted during Year 3 
at Langley Research Center. 

The results of the empirical work will be used as support for specific aspects of the com- 
putational method, or to refine and recode the method as necessary. Some of the issues 
that will certainly be addressed include defining the useful level of plan decomposition for 
the information management problem, and the tradeoffs associated with various levels of 
information granularity. Potential areas of change include: 


• Dimensions of information management 

• Plan decomposition and granularity 

• Event set 

• Information granularity 
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4 Accomplishments at Six Months 


The first six months of the project have been devoted to operationalizing our existing 
theoretical analysis of the relationship between displays and plans, with the overall goal of 
determining the appropriateness of plan-based information management. This included an 
extension of the methods, and the development of an appropriate empirical methodology. 


4.1 Laboratory Preparation 

This task involved the installation of a SPARC2 graphics engineering workstation at SUNY 
Buffalo, including connection to the local network, and printers and the installation of 
Unix, Common Lisp, C, and Hoops graphics software. The Graduate Research Assistant 
learned to program the graphics software for use in the development of the experimental 
data presentation software, and began programming some of the functions required for 
display management. The display generation capability at 6 months creates parameterized 
presentation elements and logical devices, assigns data sources to these objects, updates the 
data source according to a simple function, and allows for the manipulation of presentation 
elements according to Scope and Resolution parameters. 


4.2 Analysis of Existing Data 

4.2.1 Demonstrate the existence of plans 

Existing data were potentially informative about the feasibility of plan-based information 
presentation, and the specific scenarios in which feasibility might be easily and clearly 
demonstrated. 

Three researchers willing to provide existing data were identified: 1) Dennis Beringer, at 
New Mexico State University, who has been examining alternative 3d and 2d means of en- 
coding altitutde information for collision aviodance; 2) Phil Smith, at Ohio State University, 
who has been examining crew information requirements during replanning scenarios; and 
3) Bill Corwin, at Honeywell Systems and Research Center, who, along with V. Shalin and 
J. Bloomfield, conducted verbal protocol studies of information use during various take-off, 
landing and engine failure scenarios. Honeywell provided transcripts of its verbal protocols, 
and these were used in formulating an initial plan and goal graph for the studied portions 
of the commercial aviation task domain. 
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4*2.2 Method 


Subjects — Four training pilots from McDonnell Douglas participated in the study as part 
of their assigned work. The same pilots typically served as instructors to airline pilots. One 
of the experimenters (B. Corwin) served as a First Officer. A McDonnell Douglas employee 
served simultaneoulsy as the simulator operator, Air Traffic Control and Ground Control. 

Apparatus — The study was conducted in a fixed-base, six-degree of freedom simulator of 
the MD-80, operated for rental by FlightSafety International for training purposes. 

Procedure — Each pilot performed several take-off, level flight, engine failure and landing 
episodes, in the simulated Los Angeles Airport environment under night conditions. In a 
manipulation unrelated to the purpose of this paper, some of the flight instruments were 
obscured from pilot view. Two experimenters, in addition to the First Officer, attended 
the experimental sessions. Pilots were instructed to “think-aloud” and all flight deck events 
were audio-recorded during a two-hour experimental session (Bainbridge, 1974; Ericcson & 
Simon, 1986). This audio record was transcribed and analyzed. 


4.2.3 Analysis Sz discussion 


Our analysis of the data collected by Bloomfield, Shalin & Corwin (1990) is encouraging. 2 
A portion of the plan and goal graph developed from these data was presented earlier, in 
Figure 1. The remainder of the plan and goal graph is presented below in Figures 5a, b and 
c. 

These figures illustrates several of the domain properties that support the use of a plan and 
goal graph in information management. These properties include: 


• A plan abstraction hierarchy 

• The interleaving of plans over time 

• Subactivities shared by higher levels of abstraction 

• The activity of plan evaluation by the crew 

• The participation of multiple agents in plans 

• Requirements for plan granularity 

2 Applied Systems Intelligence, Inc., is developing a protocol analysis software tool to perform the plan- 
goal graph interpretation of protocols. 
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Figure 5a. The plan and goal graph for commercial aviation (cont.) 



Figure 5b. The plan and goal graph for commercial aviation (cont.) 


A Plan Abstraction Hierarchy — Pilot discussion can be organized according to the 
levels of abstraction of the plan-goal graph. The data illustrate two important points. First, 
pilot comments address all levels of the abstraction hierarchy. Second, several of the goals in 
commercial aviation are associated with multiple plans for achieving them (Rouse, Geddes 
k Hammer, 1990; Sewell k Geddes, 1990). Figure 6 pertains to the descent phase of flight. 
Several of the goals are associated with multiple plans for achieving them. For example, 
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the goal to Observe Speed Restriction may be achieved by using the Autopilot, by a Manual 
Power Change, or by Extending the Spoilers. Evidence for all three occurs in the protocols. 



Figure 5c. The plan and goal graph for commercial aviation (cont.) 


One of the important implications of identifying different methods for achieving the same 
goal is that the different methods potentially require different, and differently organized 
information. In this regard, note that the pilot mentions two options for accomplishing the 
goal of knowing his descent angle, by using DME- Altitude^ Rate Estimation, or Ground 
Speed- Vertical Speed Estimation: 


3 DME refers to Distance Measuring Equipment. 
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P: I know it's going to be tight 
getting down, I'd rather get 
down a little bit early than 
late, so I'm going to go ahead 
and get the spoilers out. 


P: OK. We're down to 250 so 
we've got that made. How 
are we doing on our 
restriction here? We're 
looking for 34 DME 


P: Coming up on 43, we've got 
2000 feet to lose. 

PNF: We are coming down at 2700 
feet a minute 

P: OK. 700 feet, 43, we've got 
about 10 miles, 4 miles, we've 
got about 2 mintues or so, so 
we should have made it nicely. 
Without a ground speed 
readout, we're kind of guessing 
on this. 


Figure 6. An illustration of the plan abstraction hierarchy. 


The Interleaving of Plans Over Time— The concept of a plan-goal graph is also 
important for deciding when to present information. The sequencing of conceptually in- 
dependent activities, even in well structured domains, is very difficult to script. The pi o 
protocols evidence many examples such as Figure 7, in which active goals are being ad- 
dressed from many portions of the plan and goal graph. 

The precise sequence of activity is driven by external events and convenience (Suchman, 
1987). Consequently, timely information management will require an ability to determine 
on-line exactly which plans and goals are active (Geddes, 1989). 

There is considerably more to be said about timing and information management from the 
perspective of plan-goal graphs (see below). 
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Normal 


Landing 




P: OK. There's 1 0,000 you can go ahead 
and set in 9000 now . 

' NF : OK. Set Do you want it armed ? 

P: You can arm it. 

DAC 21 contact Los Angeles 124.3. [] 

see any more airspeed restrictions, 
t to go to.... slow at 200 knots. 

approach, this is DAC 21 
\ 000 . 

AT&; DAC^troger ridar contact. Expect your 
radar^btpr momentarily. [] 

P: OK, looking fdfijhe 32, slowing towards 220 
and when we g&down thisnext 1000 feet 
well go ahead ancN&t the slstt$jjp. [] 

P: Set in 9000. Give me about // this is 

going to put us.... what's the DME at 
this point. 


Figure 7. Interleaved active goals and plans. 


Subactivities shared by higher levels of abstraction — The same plan may serve 
multiple goals in the plan and goal graph, and there is no requirement for these goals to be 
at the same level of decomposition in the graph. Figure 8 illustrates a portion of the plan 
and goal graph in which two plans serve as means to achieve two different goals. Although 
the first goal is achieved (Adjust Descent Angle), the pilot decides to retain the plan in 
service of the second goal (Observe Speed Restriction). 
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P: I'm looking at the flight director, 

I'm watching. ...got about 2,000 
feet to go. The CDI is centered 
and we still got about 3,000 feet 
per mintve descent, so we’re 
looking real good for the descent. 

PNF: We're almost to 11,000. 

P: Speed on 200, so when we level 
off it wiil start going towards 200. 
Ok. 700 feet to go. Now I am 
watching the flight director give 
me a nose-up indication warning. 

PNF: We still have the spoilers out. 

P: Spoilers out. That's OK. Well 
leave them out until we get down 
to 200 knots. 


Figure 8. Shared sub activities. 


The activity of plan evaluation by the crew — Figure 9 illustrates a portion of the 
protocol in which the crew implicitly acknowledge the existence of plans, and the need 
for appropriate conditions to invoke a particular plan for a goal. The pilot flying and the 
pilot-not-flying are considering two plans to achieve the goal of staying on the published 
course: Hand flying and AP Course Capture. The pilot flying indicates disapproval of the 
AP Course Capture plan until a certain altitude and DME has been achieved. 
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FLEUR at 8000. Any speed 
restrictions? 

Negative. 

Well just maintain 200. 

After FLEUR we have vector 
270 to intercept the localizer 
for 24 right 

OK. Set the heading [bug] 
on 270. 

Bug set Armed. 

Don't arm it 

Oh, don't arm it Sorry. 

I just did. 0 That's pushing it, 
isn't it? 


Yeah. OK, that's our 8000 
and the DME is what? 

25 

25. OK, now you can go ahead 
and hold.... or set heading. 


The participation of multiple agents in plans— Figure 10 illustrates the manner in 
which shared knowledge of a plan aids in the coordination of performance between multiple 
agents. By referencing the domain specific plan of using a Published Missed Approach, the 
pilot flying divides up the activities for that plan between himself and the pilot not flying. 



__ -P; Now, why donl you go ahead and 
set my missed approach heading 
and 111 set my missed approach 
altitude. 


Figure 10. Participation of multiple agents. 
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Requirements for plan granularity Certain aspects of the initial plan and goal graph 
had to be modified in light of the protocol data. In analyzing the discussion in illustrated 
Figure 11a, it was clear that something was missing in the plan and goal graph. The initial 
graph implicitly subsumed the goal of observing the speed restriction as part of observing 
the altitude restriction. This resulted in the lack of an obvious link for the pilot’s comment 
regarding the relationship between the Spoilers and the speed. In response to this portion 
of the protocol, the plan and goal graph was expanded to include the goal to Observe Spee 
Restriction, and an alternative plan for that goal was added, AP Speed. 


Standard 

Terminal 

Arrival 

Route 

\ (Observe | 

N Altitude J 

[Restriction } 

\ 





Vm looking at the flight director, 

I'm watching... .got about 2,000 
feet to go. The CD I is centered 
and we still got about 3,000 feet 
per mintue descent, so we're 
looking real good for the descent. 

We're almost to 1 1,000. 

Speed on 200, so when we level 
off it will start going towards 200. 
Ok. 700 feet to go. Now I am 
watching the flight director give 
me a nose-up indication warning. 

We still have the spoilers out. 

Spoilers out. That's OK. Welt 
leave them out until we get down 
to 200 knots. 


Figure 11a. Initial plan granularity. 


We note that the granularity of plans will influence the rate at which potentially disruptive 
display changes occur, so there is some advantage to keeping plans at a high level of abstrac- 
tion However, overly abstract plans could cause information to persist after its utility has 
passed. These issues are best resolved with specific experimental results, and an information 
management policy that balances the tradeoffs of display change rate accordingly. 
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Figure lib. Revised plan granularity. 


4.2.4 Comments on plan and goal graph analysis 

To the extent that alternative plans exist for the same goal , and to the extent that these 
alternatives have different information requirements , the plan-based information presenta- 
tion method was supported by these analyses. The manner in which the protocol data were 
obtained have some influence on the proposed graph. There are no doubt artifacts associ- 
ated with the experiment, the simulator, the reliability of information in the current crew 
station, and the currently availably mechanisms for accessing information in the present 
airspace system. 

Because information reliability is a persistent problem at least currently, the plan and goal 
graph should be modified to contain an abstract goal for verifying information. In addition, 
some top level plans are probably missing, and the activities at glide slope intercept should 
be expanded. 
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After the basic goals and plans are in place, an important next step is determining the 
information requirements at both interior and leaf not plans. 


4.3 Jumpseat Observations 


In addition to this review of existing data, we are also collecting our own observations of the 
commercial aviation environment by riding jumpseat on USAir. These observations suggest 
that pilots have difficulty managing information only when the situation departs in some 
way from the ideal stereotype, including for example, stress associated with bad weather, an 
unfamiliar airport, a crowded environment, unanticipated deviations, and new equipment. 
Under these circumstances the consequences of inadequately presented information become 
potentially operationally significant. 4 


4.4 ASRS Database Search 

Finally, as part of our efforts to confirm the appropriateness of a plan- based approach to 
determining information requirements, we have recently initiated a search of the ASRS 
database. We have received over 1,000 relevant reports, and analysis will proceed on these 
during the second half of the year. 


4.5 Extend Explanation of Plan-Information Relationship to Information- 
Presentation Element Relationship 

We have noted that presentation elements must be characterized in terms of the same pa- 
rameters as information elements, so that appropriate presentation elements can be selected 
for a given task. Our previous work addressed the relationship between plans and parame- 
terized information elements, but not the relationship between parameterized presentation 
elements and human information processing. We are making use of the existing literature 
to guide the development of computational rules to represent this relationship. Our ongoing 
review of the literature covers Human Factors, Psychology, Computer Science, Engineering 
and Decision Sciences. A partial list of the sources under consideration is provided in Table 

1 . 


4 This point is discussed further below in the subsection on dependent measures. 
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Table 1 

Partial List of Literature Review Sources 


ACM - Association of Computer 
Machinery 
Aerospace America 
Applied Ergonomics 
British Journal of Psychology 
Cognitive Science 
Cognition and Emotion 
Decision Support Systems 
Ecological Psychology 
Human Computer Interaction 
Human-Computer Interaction 
and Complex Systems 
Human Factors 
Human Factors Handbook 
Human Resources Research 
Organization 

IEEE Transactions on Software 
Engineering 


IEEE Transactions of Systems, 

Man, and Cybernetics 
Information and Management 
International Journal oT Man-Machine 
Studies 

Journal of Experimental Psychology 
Naecon Proceedings 
Organizational Behavior and Human 
Decision Processes 
Proceedings of CHI 

Proceedings of the Human Factors Society 
Scandanavian Journal of Psychology 
Spatial Vision 

Trie Quarterly Journal of Experimental 
Psychology 

Tasks, Errors and Mental Models (Book) 
Visible Language 


An on-line database of this literature is being developed with Paradox software. Forty 
articles have been entered thus far, addressing the following topics'. 


• The relationship between plans and information requirements. 

• The relationship between interpretation time and bandwidth. 

• The relationship between interpretation time and resolution. 

• The task conditions requiring specific values of scope. 

• Interactions between presentation elements. 

• Pilot scanning behavior. 

• Evaluation techniques for information management. 


Some of the findings are described below. 


4.5.1 The relationship between plans and information requirements 

The relationship between plans and information requirements is often addressed as a sec- 
ondary issue in research papers on other topics. In their study of instructions for operating 
devices, Kieras and Bovair (1984) concluded that useful information supports the inference 
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of steps for operating the device. Roth, Woods and Gallagher (1986) studied the behavior of 
process control operators. They concluded that experts anticipate the course of system re- 
sponses, and develop control strategies to manipulate process dynamics to their advantage, 
test system dynamics and obtain data not otherwise available. 


4.5.2 The relationship between interpretation time and bandwidth 

In a substantial study of eye movements in pilot performance, Harris and Spady (1985) 
reached a number of conclusions regarding the amount of time that data are examined 
and interpreted. These conclusions address distinctions between monitoring a state and 
controlling a state, and call attention to the role of flight mode in the examination and 
interpretation of data. Cheal, Lyon and Hubbard (1991) have demonstrated the additional 
interpretation effort required by specific features of display elements, such as line arrange- 
ments, line orientation and the need to disengage attention. 


4.5.3 The relationship between interpretation time and resolution 

Yntema (Yntema, 1963; Yntema and Mueser, 1960; 1962) conducted studies of memory 
span performance that are widely acknowledged to be well ahead of their time. His primary 
concern was the tradeoff between number of variables and the number of states, when 
designing information displays. The conclusion from this work is that fewer variables with 
many states is preferable to many variables with fewer states. This conclusion is relevant 
to both resolution and scope decisions. 

Keinan, Friedland and Arad (1991) suggest that subjects under stress naturally sort infor- 
mation into fewer, larger groups. 


4.5.4 The task conditions requiring specific values of scope 

Hanson, Payne, Shively and Kantowitz (1981) and Spenkelink (1990) both note the disad- 
vantage of digital displays for recovering trend information. In our analysis of information 
requirements, we would make a distinction between state information and trend informa- 
tion. Some tasks require both kinds of information about a variable, and some tasks require 
only one or the other. 


4.5.5 Interactions between presentation elements 

Wrolstad (1976) and Wendt (1982) provide general discussions of the advantages of picto- 
rial and typographic displays. Of course, the important issue is the interaction between 
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the various display elements, and the task in question (Byblow, 1990; Morris & Jones, 
1990; Sorkin and Woods, 1985). The significance of this issue for information management 
concerns the process of assigning a presentation element for an information source. The 
existing presentation element evaluation algorithm does not include the interaction of the 
candidate presentation element with other presentation elements. 


4.5.6 Pilot scanning behavior 


Harris et al. noted the situation dependent nature of instrument scanning. This supports 
the basic idea of information management. However, they also note that scanning is centered 
around a home base. Marcus (1974) suggests that standard display entry points facilitate 
processing time. The possibility of ritualized scanning behavior is a potential concern for 
information management, because the changes in displays may interrupt a highly automatic 
skill. 


4.5.7 Evaluation techniques for information management 


Lundberg (1990) suggests that the evaluation of expert systems and information retrieval 
systems are related problems. Completeness and precision are the important dimensions, 
and the amount of search activity is the suggested dependent measure. Purcell and Coury 
(1991) caution that the evaluation of alternative displays is sensitive to the order in which 
they axe learned. This has implications for both specific experimental designs, and the 
carryover of pilot experience with traditional displays. 


4.6 Experimental Work: Develop Predictive Methods 


4.6.1 Experimental design 


Based on the existing plan-and-goal graph, we have selected the area of observing speed 
restrictions while executing a Standard Terminal Arrival Route for our initial experimental 
work. The purpose of this experimental work is to test specific hypotheses of plans and 
associated information. 


Design and hypotheses — We are currently constructing alternative versions of dis- 
plays for three suitable plans for observing speed restrictions. Each display will be tailored 
for a particular plan, by manipulating the contents of the information as well as the pa- 
rameters governing its presentation and management. A supporting computational system 
will substantiate the appropriateness of each display for its own plan, as well as the relative 
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inappropriateness of a display for a sibling plan. The hypotheses associated with this design 
are summarized in Figure 12. We expect to see a relative facilitation for the display used in 
the context of its intended plan, and reduction in performance quality when less appropriate 
displays are used. We have developed a Three-by-Three design so that we could examine 
the performance impact of different amounts of departure from the display defined by the 
methods. 


Display Tailoring 


Plan 

Manual 

Control 

Use 

Autopilot 

Use 

Spoilers 

Manual Control 

+ 

0 

0 

Use Autopilot 

0 

+ 

0 

Use Spoilers 

0 

0 

+ 


+ Indicates conditions in which the plan and display are aligned 
0 Indicates conditions in which the plan and the display are not aligned 

Figure 12. Alternative displays crossed with alternative plans. 


Dependent measures— We have a sufficiently precise hypothesis to permit the use of 
a quantitative dependent measure, and have chosen response time, with a secondary task 
imposed primarily to mitigate ceiling effects. Response time correlates with error in gener , 
and in particular in this domain, as illustrated by the analysis in Figure 13 


Best Case 


Typical Case 


Visual Symbol Access 

Deliberate Operation 

Lag after Control Action 

Completion of Contol 
Action 


-10 msec 
-10 msec 
-2500 msec 
-1000 msec 

3,520 msec 


Visual Symbol Access x 

Double Composed 
Deliberate Operation 

Lag after Control Action 

Completion of Control 
Action 


10 


-100 msec 

-10,000 msec 
-2,500 msec 
-1.000 msec 
13, 600 msec 


Minimum distance to react « Total combined speed # Total Time 

= 1460 ft/sec * 3.53 sec - 1 mile 
Maximum distance to react = 1460 ft/sec * 13.6 sec - 3.5 mile 

Figure 13. Potential operational significance of delayed response time. 
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4.6.2 Conduct experiment 

Procedure — The experiment will be conducted in a fixed-based simulation environment, 
at SUNY Buffalo using HOOPS graphical programming language running on a SUN Sparc2 
through SunView. Pilot responses will be video and audio-taped, and pilot inputs into the 
simulation environment will capture any additional features of the context. 

Subjects — We have requested the assistance of USAir, the Air National Guard, and ALPA 
in locating line pilots for use as subjects. Approximately 9 subjects will be required, crossing 
three plans with three displays. Each subject will participate in an experimental session of 
not longer than 2 hours. 

Expected Results — We expect the experiment to demonstrate facilitated task perfor- 
mance based on the use of method-based displays. Moreover, performance with alternative 
displays should degrade in proportion to their departure from the method-based. This will 
illustrate that plan-based information management has the potential to enhance perfor- 


mance. 
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5 Outcomes &: Anticipated Results 

5.1 Multi-year 


We expect significant practical and theoretical contributions from the proposed line of 
research. One practical contribution is the development of a partially automated method 
for use by industry in choosing an information management policy, selecting information 
requirements attributes and implementing interfaces. We expect to obtain some quantitative 
estimates of performance advantages associated with intelligent information management, 
and principled selection of information grain size and display parameters. 

The proposed research also promises to integrate basic perceptual and cognitive theory with 
the threads of applied psychology and human factors being incorporated into information 
management and interface design. Moreover, the additional cognitive theory developed for 
this applied problem in human factors is specifically intended to influence cognitive science 
in the same way that applied problems in education have influenced the development of the 
field. Specifically, we anticipate a theoretical contribution to the representation of models 
of human knowledge for tasks in a manner that links cognitive and perceptual processes 
with a real world task environment. 
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A PA and LSIR Background 


LSIR — the Lear nin g System for Information Requirements was sponsored by Wright Re- 
search and Development Center, and conducted in the context of the Intelligent Pilot Vehicle 
Interface (PVI) from the Pilot’s Associate (PA) program, developed by Search Technology 
and Lockheed under the sponsorship of DARPA and the Air Force (Rouse, Geddes & Ham- 
mer, 1990). 


A.l The Pilot’s Associate PVI 

The PVI is an intelligent subsystem that links the Pilot’s Associate system and the pi- 
lot. It determines the content, modality, and format of messages, determines pilot intent, 
manages the display resources consistent with pilot intent, and transfers his intentions and 
instructions to the aircraft systems (Rouse, Geddes & Curry, 1987). 

The inputs to display selection are estimates of pilot intent, direct input from the switches 
and touch panels in the cockpit, and knowledge of information requirements associated 
with certain plans (Howard, Hammer & Geddes, 1988). The knowledge is referred to as 
the information requirements (IR) knowledge structure. The IR knowledge structure must 
be associated with a plan name and consists of a list of sublists describing information 
elements. An information element corresponds directly or indirectly to a dimension of the 
data environment in which the PA operates. 

Display generation proceeds by first matching the information requirements (IR) knowledge 
structure to the available display elements capable of illustrating each of the parameterized 
information elements on the list of information requirements (Webb, Geddes & Neste, 1989). 
Display space is managed by mapping display elements onto predefined units of space, 
starting from the center of the display, and moving outwards, according to the importance 
value of the information. 


A. 2 The LSIR Process 

For the PA program, the IR knowledge structure associated with plans was defined through 
standard knowledge acquisition and knowledge engineering activities. On the LSIR pro- 
gram, the IR knowledge structure is being computed from a description of the plan and a 
set of rules explicitly relating plan features to information requirements and their parame- 
ters. The process of deter min ing information requirements for a plan proceeds as follows: 


Begin with a coded statement of a plan, such as the one shown below. 
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• Using this statement, and event-recognition rules, determine the presence of key events 
in the plan. The current rule set is capable of recognizing the existence of an RWR- 
use event, and two Course events: precision-fixed-horizontal-course and gross-fixed- 
horizontal-course. Also in place is much of the domain theory to recognize Long- and 
Short-range-radar-use events and the fixed vs. dynamic, horizontal vs. vertical vs. 
speed, and precision vs. nominal vs. gross distinctions in course events. Defining 
the set of useful events is a continuing research problem. Major changes to the goals 
of flying an airplane (e.g., hovering) would impact this portion of the methods for 
determining information requirements. 


Events related to selecting, specializing and transitioning the plan: 
Enemy in weapon range 
Enemy radar search 
Enemy radar track 

Potential enemy-radar-guided weapon launch 
Events related to executing the plan: 

Fixed-precision course 
Fixed-gross course 
Short-range RWR use 


Figure Al. A representation of the plan for Doppler Notch in terms of events. 


• Assert a list of Information Sources with Task Dependent values which are called for by 
each recognized event. The proper grain size and level of analysis for an information 
source is yet to be determined. The smallest possible grain size is the raw data 
reported by specific aircraft systems. Thus, much of the impact of different aircraft 
systems may be localized to this portion of the methods for determining information 
requirements. 

Task dependent values represent primitive properties of the information that are sen- 
sitive to the context of the task. An examples of a task dependent attributes of 
information is Task Significant Difference. Task Significant Difference is dependent 
upon the amount of error that a particular method executed for a particular purpose 
can tolerate. Task significant difference determines the appropriate aggregation for a 
display of a continuous quantitative scale into qualitative states that elicit the same 
operator response within a state. 

• Obtain Task-Independent attribute values for each of the ISs included by an event. 
For example, the possible Range of an information element (e.g., 0 - 50,000 foot range 
of possible altitudes) is independent of the activity or method. 
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• Use the Task-Dependent and Task-Independent values associated with each IS in each 
event as inputs to a combination calculus and derive, as outputs, Scope, Resolution 
and Bandwidth values for each source of information in that Event. The use of these 
particular output values is constrained by the information management system we are 
using. If the information management system were to change, we might be required 
to output values for other dimensions. 

Figure 4 in section 2 illustrates the influence of task dependent and task independent 
attributes on the calculation of scope, resolution, and bandwidth values. The concern 
for human perception and cognition is particularly prominent in this portion of the 
rule-base. For example, the fastest possible update rate is fixed at the fastest possible 
rate at which a human can identify and interpret data, independent of the update rate 
of the raw sensors. The pilot’s ability to predict the value of an information source 
constitutes a significant portion of the calculation, and is based in large part on signal 
detection theory. 

• Ag'regate the parameter values for the same IS resulting from multiple concurrent 
events within a plan, so that each IS is left with only one scope, resolution and 
bandwidth value. Aggregation is currently done by simply taking the largest values. 


Scope 


Resolution Bandwidth 


Flight-path-vertical angle 
Flight-path-horizontal angle 
G-loading 
Heading 

Indicated Airspeed 
Thrust 

Track-Azimuth 
Track-Bearing 
Track-ID 
Track- Range 
Angle of Attack 
Roll 

EW-mode 

EW-status 

Track-class 

T rack-radar-mode 

T rack-type 

TSD-range-scale 

T rack-altitude 

Track-heading 

Track-range-rate 

Track-speed 


4 
7 

5 

6 
4 
4 

7 
6 
1 

8 
6 
6 
3 

3 

4 

7 

5 
3 

6 

8 
8 
7 


10 

9 
6 

10 

9 
5 

10 
10 
8 
9 
8 
9 
2 
1 

5 
3 

7 
1 
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8 
9 
7 
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6 
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9 

7 
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8 

4 
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6 
9 
9 
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Figure A2. An IR knowledge structure for the Doppler Notch Maneuver. 
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A. 3 Evaluation of Plan-based Information Management and Methods 


The idea of plan-based information requirements and the associated rule set relating plans 
to information is fairly new, and its evaluation has only just begun. Some preliminary 
evaluation indicates that the parameters used in the calculus is could be reduced with no 
impact to the final calculations. 

However, without the aid of LSIR, human knowledge engineers are not consistent with each 
other. In some cases, human subjects provide very modest indications that some of the 
LSIR parameters may be governing their thinking. The general conclusion is that the task 
of determining parameter values for an information management system is too complex for 
human knowledge engineers. Other than these results, LSIR has not been subjected to 
experimental testing with pilots, its performance limitations have yet to be exposed. 

Two kinds of limitations are of concern: Theoretical and practical. Theoretical limitations 
have to do with the feasibility of methodically determining correct, complete and delimited 
information requirements, given infinite effort. Examples of potential theoretical limitations 
are: excessive, tailored assumptions about independent, concurrent context that must be 
acknowledged in constructing the information requirements list, or frame-problem issues 
in controlling the growth of potentially related information. Practical limitations have to 
do with the costs and benefits of the plan-based approach relative to standard methods. 
An example of a potential practical limitation is the depth and detail of plan specification 
required to identify only minor changes with respect to the information requirements iden- 
tified using a situation-based approach. Another example of a practical limitation is the 
extent to which feedback from users can be gracefully accomodated and integrated into the 
methods for use in new situations. Practical limitations must be weighed differently than 
theoretical limitations, because the cost of the method may be balanced by an increase in 
capabilities. For example, a claim of the plan-based approach is that it generates changes in 
information requirements resulting from changes in equipment, as long as the functionality 
of the equipment is known. This capability is virtually absent from standard methods, which 
rely on operational experience in order to identify post hoc crew information requirements 

In addition to the need for empirical evidence to evaluate the plausibility of the approach, 
much of its refinement can only be resolved with data, such as the ideal granularity of the 
information elements, or the sufficiency of the parameter set. 




